COMPUTERIZED PROCESS FOR MEASURING THE VALUE OR 
PERFORMANCE OF AN ORGANIZATION OR INTANGIBLE ASSET 



This application is a continuation-in-part of U.S. Patent Application Serial 
No. 08/418,152, filed March 20, 1995, the entire disclosure of which, including 
references incorporated therein, is incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The invention relates in general to processes used by consultants, analysts, 
and executives for evaluating intangible assets, and in particular to a system for 
providing a definitive measurement of the value or performance of a technology, an 
organization, or other intangible asset of interest. 

2. Related Art 

Systems have been known for producing a grid-type graphical representation 
of the relative value of two dependent variables. One example of such systems is 
known as the Risk Management Matrix, used by the Boston Consulting Group for 
portfolio analysis. The Risk Management Matrix uses "Relative Market Share" and 
"Growth" as the two axes on a four-quadrant grid. These terms represent cash 
generation and cash use, respectively, and thus two dependent variables are 
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represented on the axes of the grid. Such analyses are discussed in detail in "A 
Manager's Guide to Technology Forecasting and Strategic Analysis Methods," 
Stephen M. Millet and Edward J. Ronton, Batelle Press, 1991, which is 
incorporated herein by reference. Further, methods of analysis such as the Risk 
Management Matrix are qualitative, and the grid axes are normally unsealed. Thus, 
the position of an intangible asset on such grids is not subject to quantitative 
analysis. As a result of all of the above, such systems produce a grid which is 
largely judgmental and arbitrary. 

Another known system of analysis is the Blake Managerial Grid, discussed 
in detail in "The Managerial Grid," Robert R. Blake and Jane Mouton, Gulf 
Publishing Company, Library of Congress Number 64-14724, 1964, which is 
incorporated herein by reference. This grid is used to display different management 
styles, with one axis representing a concern for production and the other 
representing a concern for people. Different positions on the grid are used to 
represent different management approaches, ranging from autocratic to highly 
permissive. However, The Blake Managerial Grid, like the Risk Management 
Matrix, results in a positioning which is not arrived at through any disciplined 
procedure, is not subject to quantitative analysis, and is largely judgmental and 
arbitrary. 



SUMMARY OF THE INVENTION 



It is therefore an object of the invention to provide an improved system for 
measuring the value or performance of a technology, an organization, or other 
intangible asset of interest. 

It is a further object of the invention to provide a computerized system for 
generating a chart showing the performance or value of an intangible asset. 

The process of the invention provides a graphical illustration of the 
performance or value of an asset by (1) establishing first and second variables related 
to the value of the intangible asset of interest, (2) establishing a series of performance 
criteria statements probative of the value of the first and second variables, (3) using a 
computer to calculate first and second total scores based upon the extent to which 
individual statements accurately describe the intangible asset of interest, (4) using a 
computer to generate a chart having a first axis relating to the first variable and a 
second axis relating to the second variable, (5) using a computer to plot a point on the 
chart, the point being located at coordinates corresponding to the first and second total 
scores, respectively, and (6) using the chart in making at least one decision regarding 
the value of said intangible asset of interest. 

The process of the invention according to a preferred embodiment provides a 
valuation grid which can be a means for performing, e.g., a quantitative comparison 
of organizations of a given class, a comparative analysis of various technological 




assets for venture capital pools, or a quantitative positioning of the level of 
achievement of an organization with respect to the vision and mission established by 
its owners. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing and other objects, features, and advantages of the invention 
will be apparent from the following more particular description of preferred 
embodiments as illustrated in the accompanying drawings. The drawings are not 
necessarily to scale, emphasis instead being placed upon illustrating principles of the 
invention. 

FIG. 1(a) illustrates a matrix of performance areas reflecting the value or 
performance of a for-profit company. 

FIG. 1(b) illustrates a matrix of performance areas reflecting the value or 
performance of a research and development organization. 

FIG. 1(c) illustrates a matrix of performance areas reflecting the value or 
performance of a university. 



FIG. 1(d) illustrates a matrix of performance areas reflecting the value of a 
technical asset. 

FIG. 2 illustrates examples of performance criteria for cell B2 of the matrix 
shown in FIG. 1(d). 

FIG. 3 illustrates a performance criteria scoring form. 

FIGS. 4a and 4b illustrate a valuation grid produced according to the process 
of the invention. 

Figure 4(c) illustrates a series of intangible assets ranked by their position on 
a valuation grid. 

FIG. 5 is a table illustrating names and definitions for grid positions in 
valuation grids for various organizations and assets. 

FIG. 6 is a graphical representation of performance area scores created 
according to the system of the invention. 

FIG. 7 is a screenshot showing an example of a user input page for data 
collection according to the invention. 
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FIG. 8 is a valuation grid showing approximately 150 individual intangible 
assets valued according to the process of the invention. 

FIG. 9 shows a table illustrating the results of a computer calculation of the 
evaluation of a technology for treating mineral ores to improve their separation. 

FIG. 10 shows a computer-generated chart illustrating a valuation grid with 
current and future chart positions. 

FIG. 11 shows a computer-generated bar chart with current ratings for 
performance clusters 

FIG. 12 shows a computer-generated bar chart with future ratings for 
performance clusters. 

DETAILED DESCRIPTION OF THE INVENTION 

The performance or value of an organization, technology, or other intangible 

asset can depend on, and be described in terms of, two or more independent 

variables. The first step in carrying out the process of the invention is to define at 

least two independent variables which best reflect such performance or value of the 

particular organization or asset under analysis. For example, the value of a 

particular technology or technical asset can be understood in terms of its commercial 

strength versus its technical strength; the value or position of a university can be 
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understood in terms of its teaching excellence versus its research excellence; the 
value of a research and development organization can be described in terms of its 
short-term performance versus its long-term performance; and, the value of a 
private sector company can be understood in terms of the strength of today's 
business versus the strength of tomorrow's business. The particular pair of 
independent variables, including but not limited to those pairs listed above, should 
be selected according to the type of organization or asset to be evaluated. 

For purposes of illustrating the principles of the present invention, the 
embodiments described herein utilize two independent variables in implementing a 
valuation; however, it will be appreciated by those skilled in the art that a number of 
variables greater than two could be utilized without departing from the spirit and 
scope of the invention. 

Once the first step of selecting at least two independent variables has been 
performed, the second step according to the process of the invention is to establish a 
series of performance criteria statements probative of the value of the first and second 
variables. This preferably begins with creation of an array, or matrix, of 
performance areas, i.e., areas which are considered to be important in evaluating 
the organization or asset. Examples of performance matrices for various 
organizations and assets are shown in FIGS, la- Id. As will be noted from the 
figures, the columns A, B, C, etc. preferably vary in either time, spatial, or 
geographical focus. On the other hand, the rows 1, 2, 3, etc., preferably reflect 
specific areas of activity or focus. 



A number of performance criteria, including performance statements, are 
then created for each cell in the performance matrix. FIG. 2 illustrates four 
performance criteria relating to the performance area in cell B2 (Proprietary 
Strength) of the performance matrix shown in FIG. 1(d). Each of the four 
statements for one criterion indicates a different level of strength in the performance 
area to which the criterion relates. As will be discussed in more detail below, the 
process of the invention according to a preferred embodiment includes a subsequent 
step in which an evaluator selects, from among the three or four statements for each 
of the criteria, the one statement which most accurately describes the organization 
or asset being assessed. 

As illustrated in FIG. 2, all of the performance criteria relating to the 
"proprietary strength" performance area are grouped together; however, it may be 
preferable in some cases to scramble the criteria so that performance criteria relating 
to one performance area are interspersed with performance criteria relating to other 
performance areas. Such scrambling can avoid bias during the subsequent selection 
step. 

The performance criteria can be defined by persons with extensive 
experience in the type of organization or asset being evaluated, or can be selected 
from a data base of previously-established matrices for similar organizations. 
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The performance statements preferably have rating levels and axis-v^eighting 
factors associated with them. With respect to rating levels, these are identified in 
FIG. 2 as rating levels 1, 2, 3 and 4 along the top row of the chart. In the example 
shown in FIG. 2, statements which reflect a low degree of proprietary strength are 
assigned a rating level of 0, statements which reflect a moderate degree of 
proprietary strength are assigned a rating level of 1, statements which reflect a high 
degree of proprietary strength are assigned a rating level of 2, and statements which 
reflect an outstanding degree of proprietary strength are assigned a rating level of 3. 
If only one statement is used for each performance criteria, it is preferable to assign 
to it the highest rating level, and a range of numbers used to reflect how close the 
organization or asset meets the conditions of that statement. For example, an 
evaluator could be given a series of single statements and be asked to rate each 
statement on a scale of 0 to 3, with a rating of zero indicating a low level of 
compliance with the statement by the organization/intangible asset and a rating of 3 
indicating a high level of such compliance. However, the four-statement method, 
as shown in FIG. 2, is preferred for most applications. 

With respect to assignment of axis-weighting factors, this is necessary for 

accurately plotting the orthogonal relationship between the two independent 

variables selected above in step 1 . Assignment of axis-weighting factors for each of 

the criteria serves to apportion the rating level of that criteria to the X-axis and Y- 

axis, respectively. The two right-hand colunms in FIG. 2 illustrate assigned 

weighting factors. The preferred method of performing the assignment is to 

apportion the value of unity between the two axes; for example, assigning a 
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weighting factor of 1.0 to the X-axis and a weighting factor of 0.0 to the Y-axis 
would indicate that the criteria contributed entirely to X-axis performance. 
Likewise, assigning 0.0 to the X-axis and 1.0 to the Y-axis would indicate that the 
criteria contributed entirely to Y-axis performance; and, assigning 0.5 to each of the 
axes would indicate that the contribution was equally divided between the two axes. 

The third step is to undertake the assessment and calculate scores for each of 
the independent variables. An evaluator is given a complete set of forms similar to 
that shown in FIG. 2, but with criteria for each of the performance areas on the 
appropriate performance matrix, and he selects (e.g., by circling) for each of the 
criteria the one statement which best describes the organization or asset being 
evaluated. The assessment may be undertaken by a range of stakeholders, such as 
the staff of an organization at various organizational levels, the owners, key clients, 
or alliance partners. 

Once the assessment has been undertaken, the results are preferably 

compiled in a form as shown in FIG. 3. In the example of FIG. 3, there are 37 

criteria for the nine performance areas shown in FIG. Id. In this particular 

example, a four-level rating was used, with a numerical rating of 0 assigned to 

statements which reflect the lowest performance in the corresponding performance 

area, a numerical rating of 1 assigned to statements which reflect a moderate 

performance in the performance area, a numerical rating of 2 assigned to statements 

which reflect a high performance in the performance area, and a numerical rating of 

3 assigned to statements which reflect an outstanding performance in the 
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performance area. In this example, the e valuators are requested to use letters A, B, 
C, and D to represent the four performance levels. The computer program converts 
these four letters to the corresponding numerical values. This procedure avoids 
disclosing the numerical values to the evaluators to further minimize bias. 

For each criteria, the rating level of the statement selected is multiplied by 
the X-axis weight to obtain the X-axis contribution, and by the Y-axis weight to 
obtain the Y-axis contribution. The X-axis contributions are summed and the Y-axis 
contributions are summed; the resulting totals are shown at the bottom right of the 
form in FIG. 3. In the example of FIG. 3, the X total is 25.0 out of a maximum of 
58.2 (19.4 times 3), and the Y total is 24.0 out of a maximum of 52.8 (17.6 times 
3). 

The fourth step is to plot the X-axis total and the Y-axis total on an 
evaluation grid, with the independent variables as axes. FIG. 4(a) illustrates such an 
evaluation grid resulting from application of the process of the invention to a 
research and development organization. The grid shown in FIG. 4(a) comprises a 
ten-point scale; thus, the total values obtained above of 25.0 for the X-axis and 24.0 
for the Y-axis correspond to plot positions of 4.3 and 4.5, respectively, on the grid. 

To make the graphical illustration shown in FIG. 4(a) more meaningful, it is 

preferable to assign names to each of the four quadrants in the evaluation grid. In 

FIG. 4(a), the quadrants are assigned the names "Specialty", "Pacesetter", 

"Commodity", and "Not Ready". FIG. 5 illustrates names and definitions 
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associated with the quadrants in four types of evaluation grids. 

A valuation grid for a technology would show the positioning of the 
technology in one of the four quadrants, e.g., "Specialty" in an upper left quadrant, 
"Pacesetter" in an upper right quadrant, "Commodity" in a lower right quadrant, 
and "Not Ready" in a lower left quadrant. At the inception of a new idea, the 
technology may be considered to lie at the origin. As the idea matures into a bona 
fide technology, it will follow a trajectory through the "Not Ready" quadrant to one 
of the other three quadrants. If the technology follows a forty-five degree diagonal, 
this indicates that technical and market developments are proceeding in parallel. If 
the market develops faster than the technology, the trajectory will move into the 
"Commodity" quadrant; conversely, if technical progress proceeds faster than 
market development, the trajectory will move into the "Specialty" quadrant. The 
position in which the trajectory terminates determines the overall merits of the 
technology. The higher the trajectory proceeds into the "Pacesetter" quadrant, the 
more likely it is that there will be a high level of commercial success. 

The four-quadrant grid is useful for obtaining an overall view of the status of 

a technology. However, a more insightftil display of the status is shown in Figure 

4b, in which the four-quadrant boundaries are replaced by three curves that are 

concentric with the point 10,10. Points that lie on one of these curves have 

different combinations of technical and commercial strength but have the same 

distance to "travel" to reach the upper right hand corner of the grid. With this 

display, zones between the curves can be described as "Embryonic", "Emerging", 
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"Developing" and "Ready for Commercialization". A factor "R" can be defined 
which is the progress a technology has achieved as it travels from inception (0,0) to 
fully developed (10,10). Mathematically this factor is equal to: 
(SQRT(200)-SQRT((X-10)"2+(Y-10)"2))/SQRT(200) 

The R factor can be used to rank a set of technologies, or any other 
intangible asset, as shown in Figure 4c. Ten technologies which have different X, 
Y values and therefore different R values are compared and sorted. 

It is also useful to create a chart showing the scores for each of the 
performance areas. Such a chart is illustrated in FIG. 6. This step provides a 
more-detailed graphical representation of the information represented in the 
evaluation grid by identifying the specific strengths and weaknesses of the 
organization or asset being evaluated. If the performance criteria have been 
scrambled as described above with regard to the third step, it may be necessary to 
unscramble the criteria in order to obtain scores for each performance area. 

The invention may be practiced by means of computer hardware and 

software. The following example shows a process which uses a computer program 

or programs for the evaluation of an actual technological asset of interest to a 

venture capital company, but it will be understood by those skilled in the art that the 

disclosed process could be applied to other evaluations, particularly those disclosed 

herein. The example below shows the complete process of the invention according 

to a preferred embodiment, including the initial steps of collecting and entering of 
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data, the computer calculation, and the production of the charts. It also includes an 
application of the invention for determining the future value of the asset in addition 
to determination of an asset's present value as discussed above. 

Data collection may be accomplished manually or via computer hardware 
and software, such as by world-wide web pages or other interface which presents 
users with performance criteria and receives user-input regarding the extent to which 
individual statements accurately describe an intangible asset of interest. FIG. 7 
provides an example of a user input page which is accessible electronically by a 
user, either by entry on or downloading from an internet web site or through direct 
e-mail transfer. In this respect, the form of FIG. 7 may be provided via an HTML 
form, a database data entry form (such as an MS Access form), or other suitable 
data entry means. 

The form of FIG. 7 provides for the entry of a) identification information 
concerning the intangible asset, in this case a proposal for funding a project 
submitted to a venture capital fund, b) the ratings of the applicant, and c) the ratings 
of five reviewers. In this example, there are seven sets of statements relating to 
seven performance areas, with four possible ratings in each ranging from a low 
performance (A) to a high performance (D). 

Once the performance criteria data is collected, calculations can be made on 
the data using computer hardware and software. Such calculations include, e.g., 
summing of scores, application of axis weighting factors, and generation of data 
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describing a valuation grid and a point thereon. A preferred method of making the 
calculations is to integrate three computer modules-- the data entry form shown in 
FIG. 7, a spreadsheet for carrying out the detailed calculations and plotting the 
results of the calculations, and a spreadsheet for comparing, ranking and plotting the 
data for a number of separate intangible assets. The spreadsheet for calculating and 
plotting and the spreadsheet for ranking and plotting may be provided in the form 
of, e.g., spreadsheets in the MS Excel format. This integration of the modules can 
be accomplished such that once the data has been entered in the data entry form, the 
subsequent calculations, plotting and printing of the results is performed 
automatically. 

The comparison and ranking of a number of intangible assets is an important 
feature of the invention. An example of this ranking is shown in FIG. 8 for 
proposals submitted to a funding agency by plotting the X and Y coordinates of the 
average results of reviewers who have examined and rated the proposals. 

The future value of an intangible asset can be determined by rerunning the 

calculation using new rating levels, determined through a code in the format x, y, z 

where x is the number of performance criteria improvement steps which the asset is 

likely to achieve if its current position is at the lowest performance level, y is the 

number of performance criteria improvement steps that the asset is likely to achieve 

if its current position is at the next highest performance level, and z is the number of 

performance criteria improvement steps the asset is likely to achieve if its current 

position is at the next highest performance level. The values x, y z are determined 
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based on the best judgment of those with experience in the development and 
commercialization of technology. Once determined, the x, y, z values are kept 
constant until experience shows a change is appropriate. This procedure will 
become apparent by inspecting the following example. It should be noted that a 
database can be established with data from repetitive applications of the method of 
the invention over time. The provision of such database permits modification of the 
initial x, y, and z values to more accurately reflect the number of improvement steps 
achieved in practice, whereby a "self-learning" attribute can be provided by the 
invention. 

FIG. 9 shows the results of a computer calculation of the evaluation of a 
technology for treating mineral ores to improve their separation. Columns 1,8, and 
14 show the performance criteria number. Column 2 is a letter chosen to represent 
the performance criteria statement selected by the evaluators (A, B, C, or D). 
Column 3 transforms the letters into a numerical performance rating (0, 1, 2, or 3). 
Columns 4 and 5 are the X-axis and Y-axis weight factors. Columns 6 and 7 are 
the calculated X and Y values for each performance criteria, which are totaled to 
produce the (X, Y) plotting coordinates. In columns 8 to 13, this procedure is 
repeated using a maximum performance rating (MS) that is calculated in columns 14 
to 19. Column 15 is a repeat of the performance rating (colunm 3). 

The code for calculating the maximum performance rating is shown in 

columns 16 to 18. Colunm 19 is the maximum performance rating, which is copied 

to column 9. As an example of this calculation, the performance rating for criteria 
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1 is 1. The code in columns 16 to 18 is 2, 1, 1, which means that if the 
performance rating is 0, the future rating will increase by 2, if the performance 
rating is 1, it will increase by 1, and if the performance rating is 2, it will increase 
by 1. Since the performance rating is 1, the maximum performance rating is 1 + 1 
= 2. At the bottom of Fig. 9, the scores are grouped into performance clusters and 
expressed as a percentage of the highest rating obtainable. 

Figures 10, 11 and 12 show the current and future chart positions and the 
current and future ratings for the performance clusters. Gate lines are shown in 
Fig. 10 to separate the chart into zones representing the maturity of the technology. 
By comparing Figs. 11 and 12, the venture capital manager is able to identify the 
areas where improved performance is required and the likelihood that this 
improvement can be achieved. 

Once a chart such as one shown in FIGS. 4(b) or 10 is created, the final step 
is to interpret the results for purposes of making decisions regarding the value of the 
intangible asset or organization. Such decisions include, e.g., whether to invest 
capital in a technology, how to develop a strategic plan to optimize an asset's future 
value, which programs to fund among competing programs within an organization, 
and whether an organization has met a level of achievement set forth in a mission 
established by its owners. 

Depending upon which evaluators have undertaken the analysis, various 
conclusions can be drawn from the results, and those conclusions can be of 
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importance to various stakeholders. Regarding the four particular types of 
organizations/assets described herein (i.e., a company, a research and development 
organization, a university, and a technology), the process of the invention provides 
quantitative information in at least the following areas. 

With respect to companies, managers of companies are expected to manage 
the current business activities of the company in an efficient and cost-effective 
manner, and at the same time ensure long-term survival of the company by 
developing nev^ business opportunities to replace maturing and aging products and 
markets; the process of the invention quantitatively identifies how well the company 
has met these two objectives. 

With respect to research and development organizations, the owners and 
managers of such organizations must meet the short-term needs of their clients, and 
also renew their intellectual capital through longer-term scientific and technological 
investigations, making the large transformations needed to create future business 
opportunities; the process of the invention will identify how successful the managers 
have been in meeting these objectives. 

With respect to universities, such institutions differ in their respective 
emphasis on research and teaching; the process of the invention makes it possible to 
measure their performance with respect to these sometimes-competing 
responsibilities and to achieve the balance desired by the Board of Governors. 
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With respect to technologies, technology commercialization is made difficult 
by the high risks involved in introducing new ideas into the marketplace. The 
process of the invention provides a measurement of the technical and commercial 
readiness of new technologies and enables investors to choose between competing 
proposals. 

While the invention has been particularly shown and described with 
reference to a preferred embodiment thereof, it will be understood by those skilled 
in the art that various changes in form and details may be made therein without 
departing from the spirit and scope of the invention. For example, a system having 
four statements per performance criteria has been illustrated above. However, in 
some applications, a different number of statements may be desired for either 
simplicity or enhanced rigor. 
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